Add GLM-5.2 B200 Dynamo-SGLang AgentX configs - #2995
Conversation
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 97ccdd4. Configure here.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline findings, I also checked the model.container: "dynamo-sglang" vs. image: lmsysorg/sglang:... mismatch flagged as a candidate issue — this matches the existing convention used by the sibling deepseek-v4 agentic recipes (e.g. disagg-gb300-*-mtp-kvoffload.yaml, agg-gb300-tp8-mtp-lowlatency.yaml), where container is a launcher alias key (mapped to the squash file in launch_b200-nscale-slurm.sh) rather than a literal image reference, so it is not a violation here.
Extended reasoning...
This run's inline findings cover the missing --no-preflight exemption for the node-local NVMe GLM-5.2 model path, the pre-existing sed that halves the new recipes' 4-hour health-check load budget to 2 hours, and the copy-pasted SGLANG_DSV4_REASONING_EFFORT env var that silently no-ops instead of the correct SGLANG_REASONING_EFFORT. Those are substantive enough that a human should review before merge, so I am not approving.
I additionally verified the ruled-out candidate about model.container (literal dynamo-sglang) differing from the master image (lmsysorg/sglang:nightly-dev-20260910-00840301). AGENTS.md's rule that model.container must equal image for image bumps is written with the direct-image recipes in mind (e.g. deepseek-v4/8k1k/*.yaml, which do set container to the literal sglang image string). The new glm5.2 agentic recipes instead follow the exact pattern already used by the sibling deepseek-v4 agentic recipes, where container: "dynamo-sglang" is a launcher alias resolved by runners/launch_b200-nscale-slurm.sh (mapped to SQUASH_FILE), not a literal image reference — so this is consistent with established convention, not a new inconsistency introduced by this PR.
Additional findings (outside the current diff — GitHub can't attach inline comments there):
-
🔴
runners/launch_b200-nscale-slurm.sh— New glm5.2/fp4 lane sets MODEL_PATH under $NSCALE_MODEL_ROOT (/scratch/models), the same node-local-NVMe-only storage that is why dsv4/kimik2.6/kimik3 are exempted from srtctl preflight via --no-preflight here, but glm5.2 was not added to that exemption list. Result: every GLM-5.2 multi-node job runs srtctl apply with preflight enabled, and its Path.is_dir() stat on model.path executes from the login node where /scratch/models isn't mounted, failing the job before sbatch submission. Fix: add$MODEL_PREFIX == "glm5.2"to the SRTCTL_PREFLIGHT_ARGS no-preflight condition alongside kimik2.6/kimik3/dsv4.Extended reasoning...
Line 53 sets MODEL_PATH="${MODEL_PATH:-$NSCALE_MODEL_ROOT/GLM-5.2-NVFP4}"; lines 19-21 document NSCALE_MODEL_ROOT as node-local NVMe, 'not a shared filesystem', pre-staged only on compute nodes. Lines 368-372 build SRTCTL_PREFLIGHT_ARGS and add --no-preflight only for kimik2.6/kimik3/dsv4 ('These weights are staged on the Slurm compute nodes, not the login node'), the exact rationale that also applies to glm5.2's identical NSCALE_MODEL_ROOT path. Comments in launch_gb200-nv.sh:734-736 and launch_gb300-nv.sh:543 confirm srtctl's preflight is an in-process Path.is_dir() stat run before sbatch, on the submission host. Since glm5.2 isn't in the skip list, srtctl apply at line 374 runs with preflight enabled for glm5.2 jobs, the stat fails against the unmounted /scratch/models path, and the job never reaches sbatch.
Verification: normal. The new glm5.2/fp4 lane sets MODEL_PATH=/scratch/models/GLM-5.2-NVFP4 (launch_b200-nscale-slurm.sh:52-55), where NSCALE_MODEL_ROOT=/scratch/models is documented as node-local NVMe, "not a shared filesystem," pre-staged only on compute nodes (lines 19-21). The preflight exemption at lines 366-372 adds --no-preflight only for kimik2.6, kimik3, dsv4 ("These weights are staged on the…
-
🔴
runners/launch_b200-nscale-slurm.sh— New GLM-5.2 B200 recipes set health_check.max_attempts: 1440 (4h load budget) but this pre-existing sed unconditionally rewrites it to 720 (2h) before every b200-nscale job, silently halving the load-time budget the recipe author explicitly chose for this larger model. All prior agentic recipes on this runner (kimik3, glm5.1) already used 720, so this override was a no-op until now; GLM-5.2 is the first case where it actually clobbers a larger, intentional value. Fix: only overwrite max_attempts when it is smaller than the recipe's own value (e.g. take the max), or drop the override for agentic recipes that already declare their own health-check budget.Extended reasoning...
runners/launch_b200-nscale-slurm.sh:138-149 (new) points CONFIG_FILE at e.g. agg-b200-tp8-c1-mtp.yaml, whose health_check block (lines 98-100) sets max_attempts: 1440, interval_seconds: 10 -- a 4-hour budget the recipe comment implies is needed for this model's load. Line 362 (pre-existing, untouched by this diff) runs sed -i 's/^ max_attempts: [0-9]*/ max_attempts: 720/' on CONFIG_PATH, which matches the exact two-space-indented 'max_attempts: 1440' line and rewrites it to 720, halving the budget to 2 hours before srtctl ever reads the file. If GLM-5.2's model load genuinely needs more than 7200s (as the recipe author's chosen value suggests), the health check will time out and the job will be marked unhealthy/fail even though the base cluster script never previously overrode a larger value like this, because every earlier agentic recipe on this runner already used 720.
Verification: normal. New GLM-5.2 b200-fp4 recipes deliberately set
max_attempts: 1440(two-space indent) — confirmed in all five files: agg-b200-tp8-c1-mtp.yaml:99, agg-b200-tp8-c4-mtp.yaml:99, agg-b200-tp8-c8-mtp.yaml:99, disagg-b200-1p1d-dep8-dep8-c64-mtp.yaml:169, disagg-b200-1p4d-dep8-tp4-c48-mtp.yaml:161. configs/nvidia-master.yaml wires glm5.2-fp4-b200-dynamo-sglang-agentic-agg/-disagg to runner…
| SGLANG_USE_MESSAGE_QUEUE_BROADCASTER: '0' | ||
| SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK: '0' | ||
| SGLANG_DEFAULT_THINKING: '1' | ||
| SGLANG_DSV4_REASONING_EFFORT: max |
There was a problem hiding this comment.
🔴 New GLM-5.2 B200 recipes set SGLANG_DSV4_REASONING_EFFORT: max instead of the generic SGLANG_REASONING_EFFORT that every other existing GLM-5.2 recipe (gb200, gb300) uses — a copy-paste artifact from the DeepSeek-V4 templates. Fix: rename the env var to SGLANG_REASONING_EFFORT in all 5 new recipe files (both agg single-block and disagg prefill/decode blocks) to match the established GLM-5.2 convention.
Extended reasoning...
The recipe's own comment says acceptance is pinned to a golden AL measured with reasoning forced to 'max' (thinking-on), but sglang for GLM-5.2 reads SGLANG_REASONING_EFFORT, not the DSV4-specific variable copied in here. Because the wrong-named var is silently ignored (no error), the server runs with default (not forced-max) reasoning effort, so the actual acceptance length/output distribution diverges from the golden calibration the SGLANG_SIMULATE_ACC_* settings assume, silently skewing the new curve's throughput/latency numbers relative to what the dashboard expects for this framework identity. All 5 new files (agg c1/c4/c8 and both disagg configs) have this same wrong var name at both prefill_environment and decode_environment (or aggregated_environment) blocks.
Verification: normal (new-file correctness). The 5 new GLM-5.2 b200 recipes set the DeepSeek-V4-specific env var SGLANG_DSV4_REASONING_EFFORT: max (agg-b200-tp8-c1-mtp.yaml:76, c4:76, c8:76; disagg-b200-1p4d-...-c48-mtp.yaml:56,79; disagg-b200-1p1d-...-c64-mtp.yaml:56,79) instead of the generic SGLANG_REASONING_EFFORT that every pre-existing GLM-5.2 recipe uses (e.g.… | normal. The 5 new GLM-5.2 B200…
…o-sglang-agentic-upstream # Conflicts: # perf-changelog.yaml
|
[by Codex] Addressed the review findings in 69a3ba9:
Validation: all five recipes load through srtctl, 297 matrix tests pass, launcher syntax passes, and changelog generation produces the intended Dynamo-SGLang concurrency points 1/4/8/48/64. |
…o-sglang-agentic-upstream # Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34562549528 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34580793856 |
3 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34580793856 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34580793856 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=34580793856 |
|
/reuse-sweep-run 34580793856 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — |
4 participants
Add a GLM-5.2 NVFP4 B200 Dynamo-SGLang AgentX curve on the NScale runner.
The curve includes Pareto aggregate TP8 points at concurrency 1, 4, and 8, plus disaggregated configurations at concurrency 48 and 64. All points use the same Dynamo-SGLang framework identity for dashboard grouping.
The recipes use the 2026-09-10 SGLang nightly, bundled Dynamo 1.5.0.dev20260909, EAGLE speculative decoding, and 110 GB HiCache write-back DRAM offload.
Note
Low Risk
Changes are benchmark recipes, sweep metadata, and Slurm launcher wiring only; no application runtime or auth logic is modified.
Overview
Adds GLM-5.2 NVFP4 agentic-coding benchmark coverage on B200 NScale via Dynamo + SGLang: five new
srt-slurmrecipes (aggregated TP8 at concurrency 1/4/8, disaggregated 1P(DEP8)+4D(TP4)@48 and 1P(DEP8)+1D(DEP8)@64) with EAGLE MTP, HiCache write-back, andagentic_srt.shharness wiring.Registers
glm5.2-fp4-b200-dynamo-sglang-agentic-aggand-disagginnvidia-master.yaml, documents the curve inperf-changelog.yaml, and extendslaunch_b200-nscale-slurm.shto stageglm5.2checkpoints, allowdynamo-sglang+mtp, copy recipes from a pinned srt-slurm v1.0.53 tree, skip login-node preflight, and only bump health-checkmax_attemptsto 720 when the recipe requests less (preserving GLM’s 1440×10s load budget).Reviewed by Cursor Bugbot for commit 9968d3e. Bugbot is set up for automated code reviews on this repo. Configure here.